Skip to content

Gate releases on vendored template integrity - #514

Draft
jdchawla29 wants to merge 3 commits into
jaideep/templates-cifrom
jaideep/templates-integrity
Draft

Gate releases on vendored template integrity#514
jdchawla29 wants to merge 3 commits into
jaideep/templates-cifrom
jaideep/templates-integrity

Conversation

@jdchawla29

@jdchawla29 jdchawla29 commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #513.

Adds .github/workflows/templates-integrity.yml, a reusable workflow that for each environments/ template: installs its pinned deps, overrides hud with either the repo checkout (nightly, manual dispatch) or the built wheel (source: wheel), runs the template's test suite, and builds its Dockerfile.hud image.

release.yml now runs it as a template-gate job with source: wheel and deploy depends on it — a wheel that breaks a canonical template, or a template image that no longer builds, cannot be published. The nightly schedule surfaces the same drift between releases instead of on release day.

Notes:

  • The coding template's suite already asserts oracle/null grading (golden=1.0, baseline=0.0) through a real local rollout, so the gate covers the serve/connect/run/grade path against the exact release artifact.
  • The Docker builds catch image rot; an in-container cua oracle run (scripted desktop actions over rfb) is a follow-up once the runner story for it is settled.
  • Template Dockerfiles install hud from PyPI at image build time, so the gate's docker step exercises the previous release inside the image; the wheel-override applies to the test suite. Aligning the image install with the release candidate is part of the later hud init vendoring work.

Note

Low Risk
Changes are limited to GitHub Actions orchestration; no runtime SDK or application code is modified.

Overview
Release publishing is gated on vendored environments/ templates: release.yml adds a template-gate job that calls the new workflow with source: wheel, and deploy now needs it so PyPI publish only runs after the gate passes.

The new templates-integrity.yml workflow runs on a nightly schedule, manual dispatch, and as a reusable workflow_call. For each matrix template (coding, cua), it syncs pinned deps, installs hud from the checkout (editable) or from dist/hud-*.whl when source is wheel, runs pytest, and builds Dockerfile.hud.

Nightly runs surface SDK–template drift between releases; on release, the gate ensures a wheel that breaks canonical template tests or image builds cannot ship.

Reviewed by Cursor Bugbot for commit 713acdf. Bugbot is set up for automated code reviews on this repo. Configure here.

HUD-2276

Bring the coding (v6 rebuild) and cua starter templates into the SDK repo
as canonical examples of v6 environment authoring. Each is a standalone uv
project with its own lockfile, Dockerfile, and tests.

environments/* stays ignored except the vendored templates, so local
scratch checkouts are unaffected. The templates are excluded from the
built wheel/sdist; their nested ruff config keeps root lint isolated,
and root pyright/pytest scoping already excludes them.

Follow-ups: CI matrix running template tests against the local SDK,
release-gate integrity runs, vendored hud init presets, and mirroring
back to the standalone template repos.
Each environments/ template installs its pinned deps, overrides hud with
the checkout via an editable install, then runs its own ruff and pytest.
Catches SDK changes that break the canonical templates at the PR boundary.
New reusable templates-integrity workflow: install each environments/
template with hud overridden by the checkout (nightly/manual) or the
built wheel (release gate), run its test suite, and build its Docker
image. release.yml publishes only after the gate passes against the
exact wheel being released; the nightly schedule catches drift between
releases.
@jdchawla29
jdchawla29 marked this pull request as draft July 27, 2026 03:21
@jdchawla29
jdchawla29 force-pushed the jaideep/templates-ci branch from 8c98a9c to 5b253c2 Compare July 31, 2026 02:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant